Method for magnifying images and videos

ABSTRACT

This invention is a method applicable to an image processing device, which includes the steps of providing a preprocess module for extracting a high-frequency portion of an image inputted into the device, extracting a gradient of the image and decomposing the image into plane and edge regions according to a predetermined fixed threshold, and providing a composite up-scaling module for executing the magnification processes on the image and the high-frequency portion thereof respectively, wherein the magnification process of plane regions of the image and the high-frequency portion is based on a simple interpolation while the edge regions of the image and the high-frequency portion is based on both a smart interpolation and the simple interpolation. The magnification results of the image and the high-frequency portion are then processed by a fusion process, so as to output an image having sharp but not blocky edges, rich details and strong contrast.

FIELD OF THE INVENTION

The present invention relates to a method for magnifying images andvideos, more particularly a method applicable to an image processingdevice and capable of magnifying a low resolution digital image inputtedinto the image processing device and then outputting and displaying themagnified digital image having higher clarity, better visual effect,sharper edges, less blocky edges, richer details and stronger contraston a high resolution display device.

BACKGROUND OF THE INVENTION

Recently, various digital photographing devices (such as digital camerasand video cameras) are continuously developed and improved. With theirconstantly enhanced image quality, ever-reducing product size, andsteadily lowering prices, the digital photographing devices are more andmore popular, and become indispensable tools for many people in theirdaily lives and at work.

For example, a mobile phone having a photographing function is generallyprovided with a CCD or CMOS image capturing unit for capturing images,and a small LCD screen for showing the captured images to a user.Generally, when the mobile phone is used to photograph a scene, theimages taken are stored in a memory card inserted in the mobile phone.Then, when the user selects an image listed on the LCD screen of themobile phone, the mobile phone reads the selected image from the memorycard and re-encodes the selected image into a minified image (i.e., athumbnail) for display on the LCD screen, allowing the user to executevarious operations (such as minification, magnification, drag, orpage-size adjustment) to the minified image on the LCD screen. Normally,the size of an image stored in a digital photographing device may beseveral megabytes (such as 1.2 MB) or hundreds of kilobytes (such as 120KB), and the image is stored in a storage device (such as a memory card,a hard disk, or a flash drive) of the digital photographing device. Oncethe user selects the image, the digital photographing device reads theimage from the storage device and re-encodes the original image into aminified image having a smaller size of about tens of kilobytes (such as75 KB) or several kilobytes (such as 7.5 KB) for display on a small LCDscreen, so that the user can browse the minified image and executevarious operations (such as minification, magnification, drag, orpage-size adjustment) to the minified image according to actual needs.

As the image resolution and the capturing speed of digital photographingdevices are continuously increased, a variety of digital photographingdevices have been widely used in various professional fields includingcriminal investigation, biological research, medical science, astrology,etc., for preserving important evidence, such as key clues in criminalcases, exhibits for use as evidence, and images at crime scenes; newfindings or experimental results in biological science; and medicalX-ray images, computerized tomography images, and other data helpful fordiagnosis by medical workers. Therefore, it is an important issue forresearchers in each professional field to figure out how to preservecritical evidence in laboratories or in other research fields and savethe evidence in a digital image format, so as to facilitate review orcomparison of important data in subsequent experiments or researches. Itis also important to effectively decrease the distortion of digitalimages during magnification, so that the digital images displayed havehigh resolution and are easy to identify, thereby enabling analysis anddetermination of characteristics shown in the digital images.

Furthermore, after conducting researches on the technologies formagnifying images and videos for years, image processing professionalsand relevant designers have developed various new theories andapplications continuously, from the initial linear magnification to thelater edge-based magnification, wherein frequently used linearmagnification techniques include bilinear interpolation, bicubicinterpolation, Lanzcos algorithm, etc., while a typical example ofedge-based magnification is the new edge-directed interpolation (NEDI).However, there are still some shortcomings in the foregoingmagnification techniques. For example, the linear magnification tends tocause a blocky-edge effect, loss of details, and blurry edges. On theother hand, the edge-based magnification, in which interpolation isexecuted along image edges along a gradient direction to partly solvethe problems of blocky or blurry edges, tends to produce incorrectinterpolation results, especially in an image region having rich detailsand messy edges, due to an inaccurate edge direction as the edge-basedmagnification is highly dependent on the accuracy of the edge direction.Moreover, the edge-based magnification relies on a considerable amountof calculation to maintain the accuracy of results, but the efficiencyof image processing may be lowered accordingly. Finally, since most ofthe traditional magnification techniques use weighted summation ofneighborhood pixels to perform interpolation, in which the weightedsummation produces a low-pass filtering effect on the original image,some sharpness and detail information (i.e., the high-frequency portion)of the original image is inevitably lost after the original image ismagnified. Therefore, in order to recover the quality of the originalimage, image processing professionals usually perform certainenhancement and restoration processes on the magnified image. However,the enhancement and restoration processes result in new defects such asovershoot and the ringing effect.

Therefore, it is important for image processing professionals andrelevant designers to develop a new technology for magnifying images, sothat when low-resolution images and videos are magnified and shown on ahigh-resolution video-frequency apparatus, clear digital images aredisplayed to facilitate identification of characteristics shown in theimages.

BRIEF SUMMARY OF THE INVENTION

In order to overcome the shortcomings of the aforementioned traditionaltechniques for magnifying images, the inventor of the present inventionconducted extensive research and finally succeeded in developing amethod for magnifying images and videos as disclosed herein.

A primary object of the present invention is to provide a method formagnifying images and videos, which is applicable to an image processingdevice and whereby the image processing device outputs a magnifieddigital image according to a predetermined magnification ratio after adigital image is input into the image processing device. The methodincludes the steps of: providing a preprocess module and providing acomposite up-scaling module, wherein the preprocess module executes ahigh-pass filtering process on the input image for extracting ahigh-frequency portion thereof, which high-frequency portion is used toperform high-frequency compensation on the magnified image, and thepreprocess module further executes an image decomposition process on theinput image for extracting a gradient of the input image by means of agradient operator and for decomposing the input image into plane regionsand edge regions according to a predetermined fixed threshold, whereinthe plane regions and the edge regions are labeled on the input imagemagnification process of the plane regions of the original input imageand the high-frequency portion is based on a simple interpolation, suchas bicubic interpolation, while the magnification process of the edgeregions of the original input image and the high-frequency portion isbased on both a smart interpolation and the simple interpolation. In thesmart interpolation, a magnification process is executed on pixels ofthe edge regions and the high-frequency portion through a directionalinterpolation. Following that, a confidence process is executed on aresult of the directional interpolation, wherein the result of thedirectional interpolation has higher confidence if an edge direction isclearer, and lower confidence if the edge direction is less clear. Then,the result of the directional interpolation and a result of the simpleinterpolation are processed by a weighted summation according to theconfidence previously obtained. Finally, magnification results of theoriginal input image and the high-frequency portion are obtained andprocessed by a fusion process, so as to output an output image havingsharp but not blocky edges, rich details and strong contrast accordingto the predetermined magnification ratio, thereby lowering the loadingand complexity of the operations and enhancing the speed thereof.

Another object of the present invention is to provide the foregoingmethod for magnifying images and videos, wherein the preprocess moduleexecutes a mess removing process on edge pixels in the edge regionsafter decomposing the input image into the plane regions and the edgeregions and labeling both the plane regions and the edge regions on theinput image. The mess removing process is performed by firstlyextracting a neighborhood area having a predetermined range of a certainselected edge pixel, and then calculating the number of edge pixelswithin the neighborhood area. If the number of the edge pixels withinthe neighborhood area is not equal to a predetermined number, theselected edge pixel is deleted so as to remove messy edge pixels withinthe neighborhood area.

A further object of the present invention is to provide the foregoingmethod for magnifying images and videos, wherein the preprocess moduleexecutes a morphological dilation process on the edge regions aftercompleting the mess removing process, so as to expand the edge regions.

A further object of the present invention is to provide the foregoingmethod for magnifying images and videos, wherein the directionalinterpolation is executed according to gradient values of the certainselected edge pixel, so as to evaluate an edge direction thereof andthen obtain the pixels within the neighborhood area along the edgedirection for executing the directional interpolation. Furthermore, themethod further includes executing the weighted summation on the resultsof the directional interpolation and the simple interpolation in theneighborhood area, in order to increase the robustness of thedirectional interpolation and thereby prevent the generation ofincorrect ghost points in regions having unclear edge directions.

A further object of the present invention is to provide the foregoingmethod for magnifying images and videos, wherein the method furtherincludes executing a sharpening process on the image, in order to solvethe problem of blurry edges in a magnified output image caused bylow-pass filtering of an interpolation amplification function used ininterpolation. Moreover, the method uses a finite impulse response (FIR)filter capable of non-linear high-pass filtering to perform thesharpening process on the output image, thereby traditionalamplification methods, the method of the present invention not onlygenerates magnified output images having higher clarity, better visualeffect, sharper edges, less blocky edges, richer details and strongercontrast, but also reduces calculation loading.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The structure and the technical means adopted by the present inventionto achieve the above and other objects can be best understood byreferring to the following detailed description of the preferredembodiment and the accompanying drawings, wherein

FIG. 1 is a schematic view of an image processing device according tothe present invention;

FIG. 2 is a detailed schematic view of the image processing deviceaccording to the present invention;

FIG. 3 is a detailed schematic view of an image decomposition processaccording to the present invention;

FIG. 4 is a schematic view of a high-resolution image obtained frominterpolation magnification according to a preferred embodiment of thepresent invention;

FIG. 5 is a schematic view of a first composite interpolation up-scalingmodule and a second composite interpolation up-scaling module accordingto the preferred embodiment of the present invention;

FIG. 6 is a detailed schematic view of a smart interpolationmagnification according to the preferred embodiment of the presentinvention; and

FIG. 7 is a schematic view of a high-resolution image obtained fromdirectional interpolation according to the preferred embodiment of thepresent invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring now to FIG. 1 in which a method for magnifying images andvideos according to the present invention is illustrated, the method isapplicable to an image processing device 1 so that the image processingdevice 1 can output a magnified digital image according to apredetermined magnification ratio after a digital image is input intothe image processing device 1. Thus, when a low-resolution input imageand video are magnified and shown on a high-resolution video-frequencyapparatus, a clear digital image can be displayed to facilitateidentification of characteristics shown in the image. In a preferredembodiment of the present invention, as shown in FIG. 1, the methodincludes the steps of providing a preprocess module 10 and providing acomposite up-scaling module 30. Referring now to FIG. 2, the preprocessmodule 10 executes a high-pass filtering process 11 on an input image soas to extract a high-frequency portion of the input image, wherein thehigh-frequency portion is used to perform high-frequency compensation ona magnification result of the input image during subsequent processes.The preprocess module 10 further executes an image decomposition process12 on the input image so as to decompose the input image into planeregions and edge regions by using a gradient operator, wherein the planeregions and the edge regions are labeled on the input image with labelswhich are used to instruct a magnification process of the original inputimage and the high-frequency portion thereof. Furthermore, the compositeup-scaling module 30 executes the magnification process on the originalinput image and the high-frequency portion thereof, respectively,wherein the magnification process of the plane regions of the originalinput image and the high-frequency portion is based on a simpleinterpolation, while the magnification process of the edge regions ofthe original input image and the high-frequency portion is based on asmart interpolation. Finally, magnification results of the originalinput image and the high-frequency portion are processed by a fusionprocess 33, so as to output an output image according to thepredetermined magnification ratio.

Since the traditional interpolation magnification techniques generallyproduce a low-pass filtering effect, high-frequency information of aninput image will inevitably be lost after the image is magnified andoutput. According to the preferred embodiment of the present inventionshown in FIG. 1, in order to prevent excessive loss of thehigh-frequency information and prepare for subsequent processes, thepreprocess module 10 executes the high-pass filtering process 11 of FIG.2 on the input image for extracting the high-frequency portion of theinput image. During the magnification process of the input image, thehigh-frequency portion and the original input image are magnified usingthe same interpolation algorithm. After the original input image and thehigh-frequency portion are magnified, both of them are processed by thefusion process 33 to compensate for a high-frequency portion of theinput image lost during magnification thereof.

Furthermore, since the human eyes are particularly sensitive to edgeregions with large gradient in an image, in the preferred embodiment ofthe present invention, the smart interpolation (such as directionalinterpolation) is used only on the edge regions, to which the human eyesare particularly sensitive, so as to perform high-precisionmagnification, thereby decreasing the operation loading of subsequentinterpolation magnification as well as simplifying and accelerating theentire operation. On the other hand, the simple interpolation (such asbicubic interpolation) is used to magnify the plane regions in theimage. Nevertheless, the foregoing approach provides a clear magnifiedimage without compromising its visual effect. Besides, since messy edgeregions in the image (such as edges of a meadow) generally have edgedirections difficult to be precisely determined, and the human eyes paylittle attention to the precision of magnification results of such messyedge regions, in the preferred embodiment of the present invention, thesimple interpolation is also applied to the mess edge regions to lowerthe complexity and operation loading of interpolation magnification. Forthis purpose, referring now to FIG. 3, when the image decompositionprocess 12 is executed on the input image according to the preferredembodiment of the present invention, the input image is firstlyprocessed by an image decomposition 121 for decomposing the input imageinto the plane regions and the edge regions in the following manner. Agradient operator is used to extract a gradient Grd(x) of the inputimage. In the preferred embodiment of the present invention, a Sobeloperator is used as the gradient operator to extract the gradient Grd(x)of the input image. Then, a fixed threshold ThreshD is set at 16according to the sensitivity of the human eyes to gradient variation,and the input image is decomposed into the plane regions and the edgeregions according to the following function (1), wherein pixels of theinput image are labeled by labels Label(x):

$\begin{matrix}{{{Label}(x)} = \left\{ \begin{matrix}{{Edge},{{if}\mspace{14mu}\left( {{{Grd}(x)} \geq {ThreshD}} \right)}} \\{{Plane},{{if}\mspace{14mu}\left( {{{Grd}(x)} < {ThreshD}} \right)}}\end{matrix} \right.} & (1)\end{matrix}$Following that, referring again to FIG. 3, a mess removing process 122is executed on messy edge pixels in the edge regions for removing themessy edge pixels, as explained below. Firstly, a neighborhood area M×Nhaving a predetermined range of a certain selected edge pixel isextracted, and then the number Nedge of edge pixels within theneighborhood area is calculated. If the number of the edge pixels withinthe neighborhood area is not equal to (more than or less than) apredetermined number, the selected edge pixel is deleted according tothe following function (2):if (Nedge>ThrH or Nedge<ThrL)remove the edge;  (2)wherein ThrL=min(M,N) and ThrH=0.8·M·N. Finally, referring still to FIG.3, in the preferred embodiment of the present invention, a morphologicaldilation process 123 is executed on the selected edge pixel to dilatethe selected edge pixel outward and thereby expand the correspondingedge region. In the preferred embodiment, the morphological dilationprocess 123 is executed on the selected edge pixel by means of across-structure element so as to expand the corresponding edge region.

Referring back to FIGS. 1 and 2, in the preferred embodiment of thepresent invention, the composite up-scaling module 30 executes amagnification process on the original input image, the plane regions,the edge regions and the high-frequency portion, respectively, wherein afirst composite interpolation up-scaling module 31 is used to executeinterpolation magnification and a gain process on the edge regions andthe high-frequency portion, while a second composite interpolationup-scaling module 32 is used to execute interpolation magnification onthe original input image and the plane regions. The first and secondcomposite interpolation up-scaling modules 31 and 32 may use anidentical interpolation magnification algorithm. In actual operation,interpolation magnification operations of the first and second compositeinterpolation up-scaling modules 31 and 32 can be simultaneouslyexecuted to reduce the entire operation loading. Finally, the fusionprocess 33 is executed on images output from the first and secondcomposite interpolation up-scaling modules 31 and 32 according to thefollowing function (3), so as to generate a high-resolution output imageaccording to the predetermined magnification ratio:HR(x)=LR(x)*Hp(x)*CUp (x)·Gain+LR(x)*CUp (x)  (3)wherein HR(x) is the high-resolution output image, LR(x) is thelow-resolution input image, Hp(x) is a high-pass filtering function usedby the preprocess module 10, CUp(x) is an interpolation magnificationfunction used by the first and second composite interpolation up-scalingmodules 31 and 32, and Gain is a constant gain factor used by the firstcomposite interpolation up-scaling module 31.

Generally, the first and second composite interpolation up-scalingmodules 31 and 32 serve mainly to magnify the input image. Thetraditional interpolation magnification techniques can only magnify theinput image at an even-power (i.e. 2^(n) times, n=1, 2, 3 . . . )magnification ratio. If it is desired to magnify the input image at anarbitrary magnification ratio, a downsampling technique must be used.Taking a magnification ratio of two for example, the function of thefirst and second composite interpolation up-scaling modules 31 and 32 isdescribed as follows. Referring to FIG. 4 for a high-resolution imageobtained from interpolation magnification, the black pixels shown in thedrawing are directly copied from the low-resolution input image, whilethe other pixels are interpolated from the black pixels. In theinterpolation magnification process of the present invention, the greypixels in FIG. 4 are calculated first, followed by the white pixels. Inaddition, according to the present invention, different interpolationmagnification algorithms are used for the edge regions and the planeregions of the original input image. More specifically, the smartinterpolation (such as directional interpolation) is used for magnifyingthe edge region, while the simple interpolation (such as bicubicinterpolation) is used for magnifying the plane regions.

In the preferred embodiment of the present invention, the firstcomposite interpolation up-scaling module 31 executes an interpolationmagnification process on the high-frequency portion output from thepreprocess module 10 according to the labels of image regions outputfrom the preprocess module 10. On the other hand, the second compositeinterpolation up-scaling module 32 executes an interpolationmagnification process on the original input image according to thelabels of image regions output from the preprocess module 10. Referringnow to FIG. 5, the first and second composite interpolation up-scalingmodules 31 and 32 may use the same interpolation magnificationalgorithm, except that magnification result of the first compositeinterpolation up-scaling module 31 is further processed by a gainprocess. Since the preprocess module 10 has labeled the edge regions andthe plane regions of the original input image with the pertinent labels,the first and second composite interpolation up-scaling modules 31 and32 executes an edge-region determination process 51 on the pixels in theimage according to the labels. A smart interpolation 52 is applied to apixel within a region labeled as the edge region, while a simpleinterpolation 53 (such as bicubic interpolation) is applied to a pixelwithin a region labeled as the plane region. Refer now to FIG. 6 for anoperational structure of the smart interpolation 52. To begin with, adirectional interpolation 521 and the simple interpolation 53 (such asbicubic interpolation) are executed simultaneously on a selected pixelin an edge region. Then, a calculation 522 of confidence and weight isexecuted on a result of the directional interpolation 521. The principlefor the calculation of confidence is that the clearer an edge directionis, the higher confidence the result of the directional interpolation521 will have; and the less clear the edge direction is, the lowerconfidence the result of the directional interpolation 521 will have.Finally, the result of the directional interpolation 521 and a result ofthe simple interpolation 53 are processed by a weighted summation 523according to the confidence obtained from the calculation 522 ofconfidence and weight, and then output.

In the preferred embodiment of the present invention, the directionalinterpolation 521 is executed according to gradient values of a selectedpixel in an edge region, so as to evaluate an edge direction of theselected pixel and then obtain other pixels along the edge direction forexecuting interpolation magnification. Referring now to FIG. 7, a greypoint x is used as an example for explaining how to use the directionalinterpolation 521 to get the interpolated pixel value, and rotate aninterpolation sample of the grey point x by 45 degrees so as to obtainthe white pixels in FIG. 7.

Firstly, referring to FIG. 7, an interpolation calculation DPx isexecuted on twelve pixels P₀˜P₁₁, in a neighborhood area of the greypixel x according to the following function (4). Since the interpolationcalculation DPx is performed along six directions and the pixels P₀˜P₁₁,are arranged in a generally circular pattern, the interpolationcalculation DPx has high precision.

$\begin{matrix}{{{DPx} = {\sum\limits_{i = 0}^{11}{a_{i} \cdot P_{i}}}},{{\sum\limits_{i = 0}^{11}a_{i}} = 1}} & (4)\end{matrix}$wherein a_(i) is a weighting coefficient of each of the 12 pixelsP₀˜P₁₁, (i.e., P_(i)) in the neighborhood area. A gradient value in eachof the six directions is calculated according to the following function(5):Dir₀ =|P ₀ −P ₃|,Dir₁ =|P ₁ −P ₂|,Dir₂ =|P ₄ −P ₇|,Dir₃ =|P ₅ −P ₆|,Dir₄ =|P ₈ −P ₁₁|,Dir₅ =|P ₉ −P ₁₀|  (5)If some of the pixels P₀˜P₁₁, in the function (5) are yet to bedetermined, the simple interpolation 53 (such as bicubic interpolation)can be used to calculate substitute values for those undeterminedpixels. Following that, the gradient values are used to calculate theweighting coefficient a_(i) of each pixel according to the followingfunction (6):

$\begin{matrix}{{{Dir}_{m\; i\; n} = {\min\left( {{Dir}_{0},{\ldots\mspace{14mu}{Dir}_{5}}} \right)}},\left\{ \begin{matrix}{{\begin{matrix}{{if}\mspace{11mu}\left( {{Dir}_{m\; i\; n} = {Dir}_{0}} \right)} & {{{a_{0} = {a_{3} = 0.5}},{other}}\mspace{11mu}}\end{matrix}a_{i}} = 0} \\{{\begin{matrix}{{if}\mspace{11mu}\left( {{Dir}_{m\; i\; n} = {Dir}_{1}} \right)} & {{a_{1} = {a_{2} = 0.5}},{other}}\end{matrix}\mspace{11mu} a_{i}} = 0} \\\begin{matrix}{{if}\mspace{11mu}\left( {{Dir}_{m\; i\; n} = {Dir}_{2}} \right)} & {{a_{4} = {a_{7} = 0.5}},{{{other}\mspace{14mu} a_{i}} = 0}}\end{matrix} \\\begin{matrix}{{if}\mspace{11mu}\left( {{Dir}_{m\; i\; n} = {Dir}_{3}} \right)} & {{a_{5} = {a_{6} = 0.5}},{{{other}\mspace{14mu} a_{i}} = 0}}\end{matrix} \\\begin{matrix}{{if}\mspace{11mu}\left( {{Dir}_{m\; i\; n} = {Dir}_{4}} \right)} & {{a_{8} = {a_{11} = 0.5}},{{{other}\mspace{14mu} a_{i}} = 0}}\end{matrix} \\\begin{matrix}{{if}\mspace{11mu}\left( {{Dir}_{m\; i\; n} = {Dir}_{5}} \right)} & {{a_{9} = {a_{10} = 0.5}},{{{other}\mspace{14mu} a_{i}} = 0}}\end{matrix}\end{matrix} \right.} & (6)\end{matrix}$After the weighting coefficients a_(i) are obtained, the result of thedirectional interpolation 521 and the result of the simple interpolation53 are processed by the weighted summation 523 according to the function(4), so as to calculate and output the result of DPx of the smartinterpolation 52.

In addition, referring back to FIG. 2, in order to increase therobustness of the directional interpolation 521 and thereby prevent thegeneration of incorrect ghost points in an edge region with an unclearedge direction, according to the present invention, the fusion process33 is further executed on the magnification results from the simpleinterpolation and from the directional interpolation of the first andsecond composite interpolation up-scaling modules 31 are 32 according tothe following function (7), so as to output a high-resolution outputimage according to a predetermined magnification ratio:HPx=(1−fMix)·SPx+fMix·DPx  (7)wherein HPx is a pixel value of the output image; SPx is themagnification result from the simple interpolation of pixels of theinput image; DPx is the magnification result from the directionalinterpolation of the pixels of the input image; and fMix is a fusioncoefficient obtained from the following function (8):

$\begin{matrix}{{{Dir}_{mean} = {\sum\limits_{i = 0}^{5}{{Dir}_{i}/6}}},\left\{ \begin{matrix}{{if}\mspace{11mu}\left( {{{Dir}_{m\; i\; n} \cdot 2} > {Dir}_{mean}} \right)} & {{{fMix} = 0.25};} \\{{else}\mspace{14mu}{if}\mspace{11mu}\left( {{{Dir}_{m\; i\; n} \cdot 4} > {Dir}_{mean}} \right)} & {{{fMix} = 0.5};} \\{else} & {{fMix} = 1.}\end{matrix} \right.} & (8)\end{matrix}$

Moreover, referring again to FIG. 2, since all the interpolationmagnification functions used in the foregoing interpolationmagnification processes produce a low-pass filtering effect, whichcauses blurry edges in a magnified image, the method of the presentinvention further includes a sharpening process 34 to be executed on theimage, in order to increase the clarity of edges of the magnified image.Meanwhile, the method of the present invention uses a non-linearhigh-pass filter (as shown in the following function (9), such as afinite impulse response (FIR) filter) to prevent the edge overshootphenomenon from occurring during the sharpening process 34:SHP(x)=Median(LocMax(x)·LocMin(x),Fir(x))  (9)wherein SHP(x) is a sharpened result; Median(LocMax(x)·LocMin(x),Fir(x)) is a function for taking median; LocMax(x) and LocMin(x) are amaximum value and a minimum value in the neighborhood area of theselected pixel, respectively; and Fir(x) is a FIR filter capable ofnon-linear high-pass filtering. Thus, by limiting a high-pass filteringresult between the maximum value and the minimum value in a partialneighborhood area, the edge overshoot phenomenon is effectivelyprevented during the sharpening process 34.

Therefore, when the method of the present invention is used to magnify alower-resolution input image, so as to produce a magnified output imagefor display on a high-resolution video-frequency apparatus, not only isthe operation loading lower than that of the traditional edge-basedmagnification techniques, but also the magnified output image showssharper but not blocky edges, richer details and stronger contrast,despite the fact that the method of the present invention involves lesscomplex operations and is performed at a higher speed. Furthermore, whenmagnifying a digital image, the method of the present inventioneffectively decreases image distortion and provides a high-resolutionoutput digital image that facilitates identification of characteristicsshown therein.

The present invention has been described with a preferred embodimentthereof and it is understood that many changes and modifications to thedescribed embodiment can be carried out without departing from the scopeand the spirit of the invention that is limited only by the appendedclaims.

1. A method for magnifying images and videos, applicable to an imageprocessing device so that, upon inputting of a digital image thereinto,the image processing device outputs a magnified digital image accordingto a predetermined magnification ratio, the method comprising steps of:providing a preprocess module for executing a high-pass filteringprocess on the input image so as to extract a high-frequency portionthereof, wherein the high-frequency portion is used to performhigh-frequency compensation on a magnification result of the inputimage; and for executing an image decomposition process on the inputimage so as to extract an image gradient of the input image by using agradient operator and decomposing the input image into plane regions andedge regions according to a predetermined fixed threshold, wherein theplane regions and the edge regions are labeled on the input image withlabels; providing a composite up-scaling module for executing amagnification process on the plane regions and the edge regions of theinput image, respectively, wherein the magnification process of theplane regions of the input image is based on a simple interpolation,while the magnification process of the edge regions is based on both asmart interpolation and the simple interpolation; for subsequentlyexecuting a confidence process on a result of the smart interpolation,wherein the result of the smart interpolation has higher confidence ifan edge direction is clearer, and lower confidence if the edge directionis less clear; and for finally executing a weighted summation on theresult of the smart interpolation and a result of the simpleinterpolation according to said confidence, wherein the compositeup-scaling module executes the magnification processes on the originalinput image and the high-frequency portion thereof, respectively; andexecuting a fusion process by which the magnification result of theoriginal input image is fused with a magnified result of thehigh-frequency portion to form an output image.
 2. The method of claim1, wherein the composite up-scaling module comprises a first compositeinterpolation up-scaling module for executing interpolationmagnification and a gain process on the high-frequency portion of theoriginal input image, and a second composite interpolation up-scalingmodule for executing interpolation magnification on the original inputimage, the first and second composite interpolation up-scaling modulesusing an identical interpolation magnification algorithm; and whereinthe fusion process is executed on images output from the first andsecond composite interpolation up-scaling modules, in which theinterpolation magnification and the fusion process generate thehigh-resolution output image at the predetermined magnification ratioaccording to the following function:HR(x)=LR(x)*Hp(x)*CUp(x)·Gain+LR(x)*CUp(x); wherein HR(x) is thehigh-resolution output image, LR(x) is the low-resolution input image,Hp(x) is a high-pass filtering function used by the preprocess module,CUp(x) is an interpolation magnification function used by the first andsecond composite interpolation up-scaling modules, and Gain is aconstant gain factor used by the first composite interpolationup-scaling module.
 3. The method of claim 2, wherein the imagedecomposition process is executed on the input image by using thegradient operator to extract the image gradient Grd(x) of the inputimage, then decomposing the input image into the plane regions and theedge regions according to the following function and the fixed thresholdThreshD based on a sensitivity of human eyes to gradient variation, andlabeling the input image with the labels Label(x):${{Label}(x)} = \left\{ \begin{matrix}{{Edge},{{if}\mspace{11mu}\left( {{{Grd}(x)} \geq {ThreshD}} \right)}} \\{{Plane},{{if}\mspace{11mu}{\left( {{{Grd}(x)} < {ThresD}} \right).}}}\end{matrix} \right.$
 4. The method of claim 3, wherein the smartinterpolation is a directional interpolation by which the magnificationprocesses are executed on pixels of the edge regions and thehigh-frequency portion, respectively.
 5. The method of claim 4, whereinthe directional interpolation is executed according to gradient valuesof a certain selected edge pixel, so as to evaluate an edge directionthereof and then obtain pixels within a neighborhood area along the edgedirection for executing the directional interpolation.
 6. The method ofclaim 5, wherein the directional interpolation is executed according tothe gradient values of a selected pixel in a said edge region, so as toevaluate the edge direction of the selected pixel and then obtain otherpixels along the edge direction for executing an interpolationcalculation DPx according to the following function:${{DPx} = {\sum\limits_{i = 0}^{11}{a_{i} \cdot P_{i}}}},{{\sum\limits_{i = 0}^{11}a_{i}} = 1},$wherein a_(i) is a weighting coefficient of each said pixel p_(i) in theneighborhood area, and the gradient values of each said pixel p_(i)along six directions are calculated according to the following function:Dir₀ =|P ₀ −P ₃|,Dir₁ =|P ₁ −P ₂|,Dir₂ =|P ₄ −P ₇|,Dir₃ =|P ₅ −P ₆|,Dir₄ =|P ₈ −P ₁₁|,Dir₅ =|P ₉ −P ₁₀|, wherein thegradient values are further used to calculate the weighting coefficienta_(i) of each said pixel according to the following function:${{Dir}_{m\; i\; n} = {\min\left( {{Dir}_{0},{\ldots\mspace{14mu}{Dir}_{5}}} \right)}},\left\{ \begin{matrix}{{\begin{matrix}{{if}\mspace{11mu}\left( {{Dir}_{m\; i\; n} = {Dir}_{0}} \right)} & {{{a_{0} = {a_{3} = 0.5}},{other}}\mspace{11mu}}\end{matrix}a_{i}} = 0} \\{{\begin{matrix}{{if}\mspace{11mu}\left( {{Dir}_{m\; i\; n} = {Dir}_{1}} \right)} & {{a_{1} = {a_{2} = 0.5}},}\end{matrix}{other}\mspace{14mu} a_{i}} = 0} \\\begin{matrix}{{if}\mspace{11mu}\left( {{Dir}_{m\; i\; n} = {Dir}_{2}} \right)} & {{a_{4} = {a_{7} = 0.5}},{{{other}\mspace{14mu} a_{i}} = 0}}\end{matrix} \\\begin{matrix}{{if}\mspace{11mu}\left( {{Dir}_{m\; i\; n} = {Dir}_{3}} \right)} & {{a_{5} = {a_{6} = 0.5}},{{{other}\mspace{14mu} a_{i}} = 0}}\end{matrix} \\\begin{matrix}{{if}\mspace{11mu}\left( {{Dir}_{m\; i\; n} = {Dir}_{4}} \right)} & {{a_{8} = {a_{11} = 0.5}},{{{other}\mspace{14mu} a_{i}} = 0}}\end{matrix} \\\begin{matrix}{{if}\mspace{11mu}\left( {{Dir}_{m\; i\; n} = {Dir}_{5}} \right)} & {{a_{9} = {a_{10} = 0.5}},{{{other}\mspace{14mu} a_{i}} = 0},}\end{matrix}\end{matrix} \right.$ wherein, upon determination of the weightedcoefficient a_(i), a result of the directional interpolation and theresult of the simple interpolation are processed by the weightedsummation, so as to calculate and output the result of DPx of the smartinterpolation.
 7. The method of claim 6, wherein the fusion process isexecuted on the result of the simple interpolation and the result of thedirectional interpolation in the first and second compositeinterpolation up-scaling modules according to the following function, soas to output the high-resolution output image according to thepredetermined magnification ratio:HPx=(1−fMix)·SPx+fMix·DPx, wherein HPx is a pixel value of the outputimage; SPx is the result from the simple interpolation of pixels of theinput image; DPx is the result from the directional interpolation of thepixels of the input image; and fMix is a fusion coefficient obtainedfrom the following function:${{Dir}_{mean} = {\sum\limits_{i = 0}^{5}{{Dir}_{i}/6}}},\left\{ \begin{matrix}{{if}\mspace{11mu}\left( {{{Dir}_{m\; i\; n} \cdot 2} > {Dir}_{mean}} \right)} & {{{fMix} = 0.25};} \\{{else}\mspace{14mu}{if}\mspace{11mu}\left( {{{Dir}_{m\; i\; n} \cdot 4} > {Dir}_{mean}} \right)} & {{{fMix} = 0.5};} \\{else} & {{fMix} = 1.}\end{matrix} \right.$
 8. The method of claim 7, wherein, afterdecomposing the input image into the plane regions and the edge regionsand labeling both the plane regions and the edge regions on the inputimage, the preprocess module executes a mess removing process on edgepixels in the edge regions by firstly extracting a neighborhood areahaving a predetermined range of a selected edge pixel, then calculatingthe number of edge pixels within the neighborhood area, and deleting theselected edge pixel if the number of the edge pixels within theneighborhood area is not equal to a predetermined number, so as toremove messy edge pixels within the neighborhood area.
 9. The method ofclaim 8, wherein the mess removing process is performed by firstlyextracting the neighborhood area M×N having the predetermined range ofthe selected edge pixel, then calculating the number Nedge of edgepixels within the neighborhood area, and deleting the selected edgepixel if the number of the edge pixels within the neighborhood area isnot equal to the predetermined number, according to the followingfunction:if (Nedge>ThrH or Nedge<ThrL)remove the edge; wherein ThrL=min(M,N) andThrH=0.8·M·N.
 10. The method of claim 9, wherein, upon completion of themess removing process, a morphological dilation process is executed onthe edge region corresponding to said selected edge pixel so as toexpand said edge region.
 11. The method of claim 10, wherein themorphological dilation process is executed by dilating the selected edgepixel with a cross-structure element so as to expand said edge region.12. The method of claim 11, further comprising: executing a sharpeningprocess on the output image using a finite impulse response (FIR) filtercapable of non-linear high-pass filtering, thereby limiting a high-passfiltering result between a maximum value and a minimum value in apartial neighborhood area.
 13. The method of claim 12, wherein thefinite impulse response filter executes the sharpening process accordingto the following function:SHP(x)=Median(LocMax(x)·LocMin(x),Fir(x)), wherein SHP(x) is a sharpenedresult; Median( ) is a function for taking median; LocMax(x) andLocMin(x) are a maximum value and a minimum value in the neighborhoodarea of the selected pixel, respectively; and Fir(x) is the FIR filter.14. The method of claim 13, wherein the simple interpolation is abicubic interpolation.
 15. The method of claim 14, wherein the gradientoperator is a Sobel operator.